Restricting movement of a mobile robot

ABSTRACT

A robot includes a body that is movable relative to a surface one or more measurement devices within the body to output information based on an orientation of the body at an initial location on the surface, and a controller within the body to determine an orientation of the body based on the information and to restrict movement of the body to an area by preventing movement of the body beyond a barrier that is based on the orientation of the body and the initial location.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of and claims priority to U.S.application Ser. No. 16/849,241, now U.S. Pat. No. 11,465,284, filed onApr. 15, 2020, which is a continuation of and claims priority to U.S.application Ser. No. 15/796,905, now U.S. Pat. No. 10,639,793, filed onOct. 30, 2017, which is a continuation of and claims priority to U.S.application Ser. No. 14/682,658, now U.S. Pat. No. 9,868,211, filed onApr. 9, 2015, the entire contents of each are hereby incorporated byreference.

TECHNICAL FIELD

This specification relates generally to restricting movement of a mobilerobot.

BACKGROUND

A mobile robot can maneuver around surfaces defined by objects,obstacles, walls, and other structures in its surroundings. In somecases, it may be desirable to restrict movement of the robot toparticular regions of its surroundings. To do this, barriers can beerected to prevent the robot from passing into restricted regions. Forexample, a beacon that is detectable by the robot can be placed in theenvironment to restrict the robot from entering the restricted regions.

SUMMARY

An example robot can identify areas of an environment that arenon-traversable even though a structural boundary, such as a wall,obstacle, or other surface, does not exist to prevent entrance intothose areas. The robot can generate a virtual barrier to preventmovement into those areas. Various techniques are described herein forgenerating such a virtual barrier.

An example robot includes a body that is movable relative to a surface,one or more measurement devices within the body to output informationbased on an orientation of the body at an initial location on thesurface, and a controller within the body to determine an orientation ofthe body based on the information and to restrict movement of the bodyto an area by preventing movement of the body beyond a barrier that isbased on the orientation of the body and the initial location. Theexample robot may include one or more of the following features, eitheralone or in combination.

The barrier can extend through a doorway, and the initial position ofthe robot can be within the doorway. The body can include a front and aback. The barrier can extend along a line that is parallel to the backof the robot. The line can be tangential to the back of the robot. Theline can intersect the body of the robot at a location indicated by avisual indicator on the robot. The barrier can include a first line thatextends parallel to the back of the robot and a second line that extendsperpendicular to the back of the robot. The initial location of therobot can place the back of the body adjacent to the first line and aside of the body adjacent to the second line. The controller can beprogrammed to restrict movement of the body by controlling the body toperform operations including rotating at an angle relative to theinitial orientation, and traversing the area of the surface along pathsthat are substantially parallel to the barrier.

The controller can be programmed to restrict movement of the body byperforming operations including generating a map that represents an areato be cleaned and designating a virtual barrier on the map that canindicate a location that the robot is prohibited from crossing. Thebarrier can be designated by designating coordinates corresponding tothe barrier as non-traversable.

The operations of determining the orientation and restricting themovement can be performed upon entry into a handshake mode. Thecontroller can be programmed to recognize the handshake mode in responseto one or more user-initiated operations on the robot.

Another example robot includes a body that is movable along a surfacebelow the body, a camera that faces upward relative to the surface,where the camera is configured to capture one or more images of markersfixed to a structure, and a controller within the body to identifylocations of the markers based on the one or more images, and to preventmovement of the body to an area of the surface that is beyond a barrierdefined by the locations of the markers at least until one or moreconditions is met. The example robot may include one or more of thefollowing features, either alone or in combination.

The markers can include infrared image markers, and the camera may be aninfrared camera. The markers can include machine-readable informationrepresenting a name of a location corresponding to the structure, a nameof the structure, or a both the name of the location corresponding tothe structure and the name of the structure. At least one of the name ofthe location and the name of the structure can be transmitted to anddisplayed on a mobile device.

The controller can be programmed to perform operations includinggenerating a map that represents at least part of the surface,identifying the markers on the map based on the locations of themarkers, storing the map in computer memory, and storing, in computermemory, data indicating to prohibit movement of the body to the area ofthe surface that is beyond the locations of the markers on the map. Thecontroller can be programmed to identify locations of the markers basedon more than one image of the markers, and to prevent movement of thebody to the area of the surface that is beyond the locations of themarkers as identified based on the more than one image. The controllercan be programmed to, upon satisfaction of the one or more conditions,permit movement of the body to the area of the surface that is beyondthe barrier defined by the locations of the image markers and to preventmovement of the body back across the barrier at least until one or moreconditions is met.

The robot can include a transmitter to communicate with a computernetwork wirelessly to send the map over the computer network to one ormore remote computing devices. The one or more conditions can includethe robot traversing at least a percentage of an area of the surfacethat is within the barrier. The one or more conditions can include therobot traversing, two or more times, at least a percentage of an area ofthe surface that is within the barrier.

An example method of generating an occupancy grid of at least part of anenvironment that is traversable by a robot includes determining, by acontroller within the robot, a location and orientation of the robotwithin the environment, and populating, by the controller, the occupancygrid with a barrier of non-traversable cells. The barrier ofnon-traversable cells is based at least on the location and theorientation of the robot.

Another example method of generating an occupancy grid for a robot in anenvironment includes detecting, by a camera of the robot, one or morefeatures of one or more removable markers on one or more structures inthe environment, and indicating, by a controller on the robot, on theoccupancy grid that a line of cells is non-traversable based on the oneor more features. The example method may include one or more of thefollowing features, either alone or in combination.

The method can include generating one or more images of the one or morefeatures, applying an affine transformation to the one or more images toproduce one or more transformed images, and confirming that the one ormore transformed images sufficiently match one or more stored images.Indicating on the occupancy grid can be performed in response toconfirming that the one or more transformed images sufficiently matchthe one or more stored images.

Advantages of the foregoing may include, but are not limited to, thefollowing. The user can control the robot and the areas through whichthe robot navigates. The robot can be restricted to areas where therobot can move freely while reducing the risk of damage to objects inthe area. In some implementations, the robot functions autonomously andthe user does not need to monitor the robot as it covers a room in orderto keep the robot out of particular areas of the room.

Any two or more of the features described in this specification,including in this summary section, can be combined to formimplementations not specifically described herein.

The robots and techniques described herein, or portions thereof, can becontrolled by a computer program product that includes instructions thatare stored on one or more non-transitory machine-readable storage media,and that are executable on one or more processing devices to control(e.g., to coordinate) the operations described herein. The robotsdescribed herein, or portions thereof, can be implemented as all or partof an apparatus or electronic system that can include one or moreprocessing devices and memory to store executable instructions toimplement various operations.

The details of one or more implementations are set forth in theaccompanying drawings and the description herein. Other features andadvantages will be apparent from the description and drawings, and fromthe claims.

DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a view of a robot in a room.

FIG. 2A shows a perspective view of a robot.

FIG. 2B shows a cut-away side view of the robot of FIG. 2A.

FIG. 3A shows a perspective view of another robot.

FIG. 3B shows a side view of the robot of FIG. 3A.

FIG. 4 is an example control system for use with mobile robots.

FIGS. 5A to 5C include illustrations and a flowchart showing a processby which a mobile robot creates an invisible or virtual barrier for therobot.

FIGS. 6A to 6C include illustrations and a flowchart showing anotherprocess by which a mobile robot creates an invisible or virtual barrierfor the robot.

FIGS. 7A to 7C include illustrations and a flowchart showing anotherprocess by which a mobile robot creates an invisible or virtual barrierfor the robot.

FIGS. 8A to 8C include illustrations and a flowchart showing stillanother process by which a mobile robot creates an invisible or virtualbarrier for the robot.

Like reference numerals in different figures indicate like elements.

DETAILED DESCRIPTION

Described herein are example robots configured to traverse (or tonavigate) surfaces, such as floors, carpets, turf, or other materialsand perform various operations including, but not limited to, vacuuming,wet or dry cleaning, polishing, and the like. The movement of theexample robots described herein may be restricted. For example, a robotmay erect a virtual barrier, which defines a boundary that the robot maynot cross. For example, a user can select a location for a virtualbarrier to prevent the robot from entering into a particular space. Asshown in FIG. 1 , the robot is positioned in a bathroom and a virtualbarrier is generated (shown in hashed squares) to prevent the robot fromentering into the bedroom. As described herein, the virtual barrier maybe created by the robot itself (e.g., based on the robot's orientationand location), or by the robot in combination with one or more elements,such as markers that are recognizable to the robot as defining a virtualbarrier that the robot may not cross. The markers can be removed afterthe robot has initially detected the markers during an initial use.Consequently, the markers need not remain in the environment forsubsequent uses of the robot.

The robot may implement other processes for creating a virtual barrier.In some implementations, the robot can record the locations of a virtualbarrier on an occupancy grid that serves as a map of the robot'senvironment, and thereby retain in memory the locations of virtualbarriers during its navigation and/or between missions. An occupancygrid can be a map of the environment as an array of cells ranging insize from 5 to 50 cm with each cell holding a probability value (e.g., aprobability that the cell is occupied) or other information indicativeof a status of the cell. The occupancy grid can represent a map of theenvironment as an evenly spaced field of binary random variables eachrepresenting the presence of an obstacle at that location in theenvironment. While some of the examples described herein use anoccupancy grid to provide the robot with a map of the environment, othermapping techniques could be used. For example, a different maprepresentation, such as a graph, where the virtual barrier isrepresented as a line segment comprised of two or more coordinates or avirtual polygon comprised of three or more coordinates or any othergeometric shape or “lasso” shape could be used with the methods andsystems described herein.

Virtual barriers can keep a robot from exiting or entering a particulararea, e.g., to prevent a cleaning robot from moving from a bathroom areato a living room area. The virtual barriers may be temporary in that,upon satisfaction of one or more conditions, the robot may be permittedto cross the virtual barriers. For example, if a robot determines thatit has cleaned the entirety of a room, the robot may then be permittedto cross a virtual barrier located across that room's exit. In thisexample, the robot may be prohibited from crossing back into thepreviously cleaned room due to the virtual barrier (unless, e.g., therobot's charging base is located in the room).

The techniques described herein may be used to restrict movement of anyappropriate type of robot or other apparatus, including autonomousmobile robots that can clean a floor surface of a room by navigatingabout the room. An example of such a robot is floor cleaning robot 100shown in FIG. 2A. The robot 100 includes a body 102, a forward portion104, and a rearward portion 106. The robot 100 can move across a floorsurface of a physical environment through various combinations ofmovements relative to three mutually perpendicular axes defined by thebody 102: a transverse axis X, a fore-aft axis Y, and a central verticalaxis Z. A forward drive direction along the fore-aft axis Y isdesignated F (referred to hereinafter as forward), and an aft drivedirection along the fore-aft axis Y is designated A (referred tohereinafter as rearward). The transverse axis X extends between a rightside R and a left side L of the robot 100.

A user interface 110 is located on a top portion of the body 102 and isconfigured to accept one or more user commands and/or display robotstatus. The top portion of the body 102 also may include a camera 109that the robot 100 can use to capture images of the environment. Therobot can detect features in the environment based on the imagescaptured by the camera 109. The camera 109 can be angled upward relativeto a surface supporting the robot (e.g., a floor) so that the camera 109can capture images of wall surfaces of the environment. As describedherein, in some implementations, the camera 109 can detectuser-positionable and removable barrier identification markers, such asstickers or other visual identification devices on wall (or other)surfaces of the environment, and based on these barrier identificationmarkers, generate virtual boundaries that the robot 100 is instructednot to cross.

A wall following sensor 113 on the right side of the robot 100 mayinclude an IR sensor that can output signals for use in determining whenthe robot 100 is following a wall. The left side L of the robot 100 canalso have a wall following sensor of this type. The forward portion 104of the body 102 includes a bumper 115, which is used in detectingobstacles in a drive path of the robot 100. The bumper 115 and/or therobot body 102 can include sensors that detect compression of the bumper115 relative to the robot body 102, such as compression based on contactwith an obstacle. In some implementations, the top of the robot 100includes an omnidirectional infrared (IR) transceiver 118 that candetect infrared radiation emitted from objects in the environment. Thesesensors can cooperate with other user inputs to provide instructions tothe robot 100 regarding boundaries or obstacles in the environment.

Referring to FIG. 2B, a front roller 122 a and a rear roller 122 bcooperate to retrieve debris from a cleaning surface. More particularly,the rear roller 122 b rotates in a counterclockwise sense CC, and thefront roller 122 a rotates in a clockwise sense C. The robot 100 furtherincludes a caster wheel 130 disposed to support the rearward portion 106of the robot body 102. The bottom portion of the robot body 102 includeswheels 124 that support the robot body 102 as the robot 100 navigatesabout a floor surface 10. As the wheels 124 are driven, rotary encoders112 measure the position of a motor shaft driving the wheels, which canbe used to estimate the distance travelled by the robot 100.

The bottom of the robot body 102 includes an optical mouse sensor 133that includes a light source and a low-resolution camera. The robot 100can use the optical mouse sensor 133 to estimate drift in the x and ydirections as the robot 100 navigates about the environment.

The robot body 102 further houses an inertial measurement unit (IMU)134, e.g., a three-axis accelerometer and a three-axis gyroscope tomeasure (i) x, y, and z acceleration and (ii) rotation about the x-, y-,and z-axes (e.g., pitch, yaw, and roll), respectively. The acceleratorof the IMU 134 can be used to estimate drift in the x and y directions,and the gyroscope of the IMU 134 can be used to estimate drift in theorientation θ of the robot 100. These measurement devices, e.g., the IMU134, the optical mouse sensor 133, and the rotary encoders 112,cooperate to provide, to the controller, information (e.g., measurementsrepresented as signals) about the location and orientation of the robotthat the controller uses to determine the approximate location andorientation of the robot 100 in its environment. In someimplementations, these measurement devices may be combined into a singledevice or into two devices.

FIGS. 3A and 3B show another example of a mobile robot that can createvirtual barriers according to the example techniques described herein.Referring to FIG. 3A, in some implementations, a mobile robot 200 weighsless than 5 lbs (e.g., less than 2.26 kg). The robot 200 is configuredto navigate and clean a floor surface. The robot 200 includes a body 202supported by a drive (not shown) that can maneuver the robot 200 acrossthe floor surface based on, for example, a drive command having x, y,and 0 components. As shown, the robot body 202 has a square shape anddefines an X-axis and a Y-axis. The X-axis defines a rightward directionR and a leftward direction L. The Y-axis defines a rearward direction Aand a forward direction F of the robot 200. Also referring to FIG. 3B, abottom portion 207 of the robot body 202 holds an attached cleaning pad220, which supports a forward portion 204 of the robot 200. The bottomportion 207 includes wheels 221 that rotatably support a rearwardportion 206 of the robot body 202 as the robot 200 navigates about thefloor surface. Mobile robot 200 may also include an IMU, an opticalmouse sensor, and rotary encoders, as described herein, to output, tothe controller, information representing the current orientation andlocation of the robot.

The body 202 includes a movable bumper 210 for detecting collisions inlongitudinal (A, F) or lateral (L, R) directions. That is, the bumper210 is movable relative to the body 202 of the robot, and this movementmay be used to detect collisions by detecting when the bumper 210 iscompressed.

The top portion 208 of the robot 200 includes a handle 235 for a user tocarry the robot 200. The user can press a clean button 240 to turn onand off the robot 200 and to instruct the robot 200 to, for example,begin a cleaning operation or mark a virtual barrier in its occupancygrid. In some implementations, the top portion 208 also includes lights242 a and 242 b or other visual indicators aligned along a line parallelto the back side 202A of the robot body 202. The lights 242 a and 242 bcan be light-emitting diodes (LEDs). As described herein, the lights 242a and 242 b can serve as a reference line for a user to determine theplacement of a virtual barrier in an occupancy grid of the robot 200.

Referring to FIG. 4 , a robot (e.g., the robot 100, the robot 200, andother appropriate mobile robot, including those described herein)includes an example control system 300 that includes a power system 350,a drive 360, a navigation system 370, a sensor system 380, acommunications system 385, a controller circuit 390 (herein alsoreferred to as controller), and a memory storage element 395. The powersystem 350, which includes a power source, provides electric power tothe systems operable with the robot.

The drive 360 can maneuver the robot across the floor surface. The drive360 can control motors to drive wheels (e.g., the wheels 124, 221) suchthat the wheels can propel the robot in any drive direction along thefloor surface. The wheels can be differentially operated such that therobot can turn based on a level of drive supplied to each drive wheel.

The navigation system 370, which may be a behavior-based system executedon the controller 390, can send instructions to the drive system 360 sothat the robot can use the drive 360 to navigate an environment. Thenavigation system 370 communicates with the sensor system 380 to issuedrive commands to the drive 360.

In some implementations, the sensor system 380 includes sensors disposedon the robot, (e.g., obstacle detection sensors, the wheel encoders 112,the optical mouse sensor 133, the IMU 134) that generate signalsindicative of data related to features of structural elements in theenvironment, thereby enabling the navigation system 370 to determine amode or behavior to use to navigate about the environment to enablecomplete coverage of a room or cell. The mode or behavior can be used toavoid potential obstacles in the environment, including wall surfaces,obstacle surfaces, low overhangs, ledges, and uneven floor surfaces. Thesensor system 380 creates a perception of the robot's environmentsufficient to allow the robot to make intelligent decisions aboutactions (e.g., navigation actions, drive actions) to take within theenvironment. The sensor system 380 gathers the data to allow the robotto generate an occupancy grid of the environment.

In some implementations, the sensor system 380 can include obstacledetection obstacle avoidance (ODOA) sensors, ranging sonar sensors,proximity sensors, radar sensors, LIDAR (Light Detection And Ranging,which can entail optical remote sensing that measures properties ofscattered light to find range and/or other information of a distanttarget) sensors, a camera (e.g., the camera 109, volumetric point cloudimaging, three-dimensional (3D) imaging or depth map sensors, visiblelight camera and/or infrared camera), and wheel drop sensors operablewith caster wheels (e.g., the caster wheel 130). The sensor system 380can also include communication sensors, navigation sensors, contactsensors, a laser scanner, and/or other sensors to facilitate navigation,detection of obstacles, and other tasks of the robot. The proximitysensors can take the form of contact sensors (e.g., a sensor thatdetects an impact of a bumper on the robot with a physical barrier, suchas a capacitive sensor or a mechanical switch sensor) and/or proximitysensors that detect when the robot is in close proximity to nearbyobjects.

The controller 390 operates with the other systems of the robot bycommunicating with each system to provide and to receive input andoutput parameters. The controller 390 may facilitate communicationbetween the power system 350, the drive system 360, navigation system370, the sensor system 380, the communications system 385, and thememory storage element 395. For instance, the controller 390 caninstruct the power system 350 to provide electrical power to the motorsof the drive system 360 to move the robot in the forward drive directionF, to enter a power charging mode, and/or to provide a specific level ofpower (e.g., a percent of full power) to individual systems. Thecontroller 390 may also operate the communications system 385, which caninclude a wireless transceiver including a transmitter that cancommunicate with mobile devices or a central computer network. Asdescribed herein, the controller 390 may upload an occupancy gridgenerated during a cleaning operation of the robot to the centralcomputer network or individual mobile devices. The communications system385 may also receive instructions from a user.

The controller 390 can execute instruction to map the environment andregularly re-localize the robot to the map of the environment. Thebehaviors include wall following behavior and coverage behavior.

In general, during wall following behavior, the robot detects a wall,obstacle (e.g., furniture, breakfast bar, cabinet toe kick, etc.), orother structure (e.g., fireplace hearth, stair edge, etc.) in theenvironment (using, for example, the bumper 115), and follows thecontours of the wall, obstacle or other structure.

During the coverage behavior, the controller instructs the robot tocover (e.g., traverse or navigate the extent of) and to clean the floorsurface of the environment. The robot can cover the floor surface of theenvironment using coverage path techniques, such as a boustrophedon orcornrow pattern, a spiral pattern, or a pseudo-random bounce coverage.As the robot covers the floor, the controller 390 can generate anoccupancy grid.

In some implementations, the controller 390 may use, for example,information (e.g., signals) from the encoders 112, the optical mousesensor 133, and the IMU 134 to generate odometry data that can be usedto determine (e.g., to estimate) the position and orientation (pose) ofthe robot. For example, the controller can receive gyroscope signalsfrom the 3-axis gyroscope of the IMU 134. The gyroscope signals can bebased on an orientation and position of the body of the robot as therobot navigates a floor surface. The controller can also improve theestimate using signals from the encoders 112, which deliver encodersignals based on the distance travelled by the robot. Similarly, theoptical mouse sensor 133 generates signals that can be used to determinethe amount of drift of the robot as the robot navigates about the floorsurface.

The memory storage element 395 can include a mapping module 397 thatstores an occupancy grid of a room or rooms that the robot navigates.The occupancy grid can be uploaded to a remote computing device usingthe communications system 385 after a cleaning operation. In someimplementations, the occupancy grid includes a virtual map generated bythe controller 390 and used by the controller 390 to instruct the robot100 to navigate within pre-determined boundaries, physical boundaries,and other boundaries (e.g., virtual or use-established barriers orboundaries). The occupancy grid may include the physical layout of theenvironment. For example, the occupancy grid may include data indicativeof the physical layout of the area and represent both open areas andobstacles. The occupancy grid can include a boundary of the environment,boundaries of obstacles therein, boundaries generated before starting acleaning operation that may or may not correspond to physical obstaclesin the environment, and/or the interior floor space traversed by therobot.

The occupancy grid may be implemented in any appropriate manner,including without limitation, as a map of locations of properties, usingdatabase techniques, using a variety of associative data structures, orany other method of organizing data. Thus, the resulting map need not bea visible map, but may be defined via data stored in non-transitorycomputer readable memory. A map may correspond to an actual surface withdifferent degrees of precisions and/or accuracy. Precision may beaffected, for example, by the use of discrete map cells that correspondto a portion of the surface. The size of those cells, which may eachcorrespond to a 10 cm×10 cm portion of the surface, or a 5 cm×5 cmportion of the surface (for example—they need not be square or even allof the same size) may affect precision by imposing limitations on thegranularity of observed properties. Accuracy may be affected by sensorquality and the like, including various other factors mentions herein.

In some implementations, the occupancy grid is an occupancy gridincluding a 2D grid of cells with each cell having an associatedvariable indicative of the status of the area for traversal or cleaning.Each cell in the occupancy grid can be assigned a value indicatingwhether the cell is traversable or non-traversable. Each cell of thegrid can be assigned (x, y) coordinates based on a chosen origin (0, 0)cell in the environment. The chosen origin can be, for example, thecharging dock of the robot or a particular location in the room. Eachcell can represent a square area with four sides that coincide with thesides of other cells. The cells can have a side length between 1 and 100cm in some implementations. For example, the grid can be a grid ofcells, each 10 cm×10 cm. Cells of the occupancy grid can be populatedbefore a cleaning operation and during the cleaning operation. In somecases, the populated cells from one cleaning operation can be stored andused for a subsequent cleaning operation. Before a cleaning operation, asubset of cells of the occupancy grid can be marked as non-traversable.In some cases, the cells form a user-established virtual barrier thatrepresents a non-traversable boundary for the robot (e.g., the virtualbarrier may be defined by a line of non-traversable cells in theoccupancy grid). As described herein, the cells can be marked as part ofa previous cleaning operation, or the robot can receive instructions topre-populate some cells of the occupancy grid as non-traversable. Inanother implementation, the occupancy grid can be an occupancy graphwhere the virtual barrier is represented as a line segment defined bytwo or more coordinates, a virtual polygon defined by three or morecoordinates, or any other geometric shape or “lasso” shape defined bymultiple coordinates.

During a cleaning operation, the controller 390 stores the (x, y)coordinates of each cell traversed by the robot. During wall followingbehavior, for example, the controller 390 can mark all cells under thefootprint of the robot as traversable cells and mark all the cellscorresponding to the wall being followed as non-traversable to indicatethat the robot 100 cannot pass the wall. As described herein, thecontroller 390 may be configured to recognize specific sequence,combinations, groups, etc., of cells that represent features of thestructural elements in the environment (e.g., walls, obstacles, etc.).In some implementations, before determining the value of cells in themap, the controller 390 can pre-set the values of all cells to beunknown. Then, as the robot drives during the wall following behavior orduring the coverage behavior, the values of all cells along its path areset to traversable, the location of the cells being determined by thedistance to the origin. In some cases during the cleaning operation, thesensor system 380 may additionally or alternatively respond to features(e.g., markers) located in the room, and the controller 390 may indicatea virtual barrier in the occupancy grid based on sensing the features.

In addition to marking cells as non-traversable as described herein,several methods to generate virtual barriers and non-traversable cellsare also described herein. During a cleaning operation, the controllercan instruct the robot to avoid the areas designated in the occupancygrid as non-traversable. While the occupancy grid is often stored on therobot (e.g., on the memory storage element 395), the occupancy grid maybe transmitted through the communications system 385 and stored on anetwork server, a mobile device, or other remote computing device.

The examples herein describe an environment and a correspondingoccupancy grid for the environment. The occupancy grids in FIGS. 5A, 5B,6A, 6B, 7A, 8A, and 8B use hashed cells to identify non-traversableareas, the blank cells to identify traversable areas, and areas nototherwise marked with cells to identify unknown areas. The robot shownin the corresponding occupancy grid identifies the controller's estimateof the robot's current location in the environment.

While the occupancy grids described in FIGS. 5A, 5B, 6A, 6B, 7A, 8A, and8B show examples of occupancy grids that include cells to indicatetraversable and non-traversable areas of the environment, in otherimplementations, the controller can generate an occupancy grid thatrelies on coordinate values corresponding to locations within theenvironment. For example, a virtual barrier can be a set of two or moretwo-dimensional coordinates that indicate the vertices of a line orregion that the robot cannot cross.

In some implementations, the robot may execute multiple cleaningoperations to clean multiple rooms in an environment. Referring to FIG.5A, as a robot 400 navigates about the floor surface 10 of anenvironment 410 containing a first room 412 and a second room 414 (e.g.,as shown in portion 421 of FIG. 5A), the controller 390 of the robot 400generates a corresponding occupancy grid 420 (e.g., an occupancy gridstored in the memory storage element 395, as shown in portion 423 ofFIG. 5A) of the environment 410. A doorway 415 separates the first room412 and the second room 414. As described in more detail herein, therobot 400 can first clean the first room 412 and then proceed to cleanthe second room 414 without returning to the first room 412.

The robot 400 executes a cornrow pattern along a path 425. The path 425can be generally restricted to a first region 430 a. Regions 430 a and430 b may be regions of equal width that the robot 400 sets in order tosegment an environment. The regions may be arbitrarily selected andtherefore may or may not correspond to physical boundaries, obstacles,or structures within the environment.

As the robot 400 follows coverage behavior by executing the cornrowpattern along the path 425, in order to restrict itself to the region430 a, the robot 400 may stop itself from entering a region 430 b of theenvironment. The controller 390 can instruct the robot 400 to avoidentering the region 430 b and to turn around during execution of theranks of the cornrow pattern. In the occupancy grid 420, the controller390 indicates non-traversable cells that correspond to walls of theenvironment and indicates traversable cells as areas that the robot 400was able to cover during the coverage behavior.

When the controller 390 has determined that the robot 400 has been ableto cover the traversable areas of the region 430 a, the robot 400 canexecute wall following behavior to advance to another region of theenvironment 410, for example the region 430 b. The controller 390 candetermine that the robot 400 has completed covering the first region 430a by determining that the robot 400 has met one or more conditions.Referring to FIG. 5B, as shown in the portion 421, the robot 400 canfollow a path 440 to perform wall following. The robot 400 starts at aninitial position 440 a that corresponds to the position of the robot 400when it completed the coverage behavior. At a position 440 b along thepath 440, the robot 400 crosses from the first region 430 a into thesecond region 430 b. At this point, the controller 390 determines thatthe robot 400 has entered a new region. The controller 390 can make thisdetermination by, for example, determining that the robot 400 has movedfrom a traversable cell to an unknown cell. The controller 390 can alsodetermine that the robot 400 has exited the first region 430 a andentered the second region 430 b.

In order to prevent the robot 400 from returning to the region 430 a,where it has already executed a cleaning operation, the controller 390can establish a virtual barrier 450 that marks regions that the robot400 has already cleaned, as shown in the portion 423. For example, thecontroller 390 can update the occupancy grid 420 to identify a locationor boundary of the previously cleaned area to prohibit the robot 400from returning to the area. During a cleaning (e.g., non-docking)operation and/or can mark all cleaned cells in the occupancy grid 420 toprohibit the robot 400 from re-cleaning those cells during the cleaningoperation. In some examples, the controller 390 can mark perimeter cellsforming the perimeter of the room 412 as non-traversable in theoccupancy grid 420. In some cases, the controller 390 marks the cellsthat encompass the traversable cells of the region 430 a asnon-traversable to stop the robot 400 from returning to regions that therobot 400 has already cleaned. In other cases, the controller 390 canindicate all cells in the region 430 a as non-traversable.

Referring to FIG. 5C, a flow chart 460 illustrates a method for a robotto clean a first area and a second area. At operation 462, the robotexecutes a first cleaning operation in a first area. The robot canexecute the first cleaning operation in response to instructions issuedby a controller of the robot. The robot can execute a coverage behaviordescribed herein, which can include following a cornrow pattern or otherpatterns to cover the first area. As the robot performs the coveragebehavior, the controller can mark cells in an occupancy grid stored onthe robot (e.g., on a memory storage element operable with thecontroller) corresponding to portions of the first area traversed by therobot as traversable. The cleaning operation may be executed by a drycleaning robot, such as the robot 100, a wet cleaning robot, such as therobot 200, another mobile robot configured to navigate about anenvironment.

At operation 464, the robot, via the controller, determines that thefirst cleaning operation is complete. The controller can determine thecompletion based on one or more conditions described herein.

At operation 466, the robot navigates to a second area. In someexamples, the robot can traverse a perimeter of the first area toidentify the second area. In other examples, the first area may beartificially bounded (e.g., be a maximum width) and the second area canbe a region adjacent to the first area. The controller can instruct therobot to perform the navigation. Generally, the controller can seek todetermine that the robot has exited an area that it has already cleanedand has entered an area that it has not cleaned. The controller caninstruct the robot to traverse the perimeter after the robot hascompleted the cleaning operation of the first area. The controller candetermine that the robot has completed the cleaning operation based ondetecting that the robot has fulfilled one or more conditions. In somecases, the robot may continue the cleaning operation until the robot hascovered a percentage of the area of the first room, for example, 50% to75%, 75% to 100%, 100% to 150%, 150% to 200%, 250% to 300%. In somecases the robot may continue the cleaning operation until it has thearea multiple times, for example, once, twice, three times, or fourtimes. Upon completing the desired coverage, the controller may instructthe robot to cross the virtual barrier and begin a second cleaningoperation in the second room.

In some implementations, the robot may continue the cleaning operationuntil the robot has reached a certain lower limit charge percentage, forexample, 10%, 5%, or less. Upon reaching the lower limit chargepercentage, the controller can instruct the robot to return to acharging dock or charging station to re-charge a battery of the robot.In such implementations, the robot may be able to traverse virtualbarriers stored in the occupancy grid in order to return to the chargingdock.

In some cases, the first area is a room and the perimeter of the firstarea thus can correspond to walls of the room. In other implementations,the first area is a region (as described herein), and the perimeter ofthe first region may correspond to the edge of the expanse of the firstregion. As described with respect to FIGS. 5A to 5B, when the robot 400executes the wall following behavior, the controller 390 can determinethat it has traversed a perimeter of the first room 412 or the firstregion 430 a by, for example, (i) detecting that the robot 400 hasexited the first region 430 a or (ii) detecting that the robot 400 hasmoved from a traversable cell to an unknown cell. The robot can traversethe perimeter of the first area in response to instructions from thecontroller.

At operation 468, the controller establishes a virtual barrier that, forexample, separates the first area and the second area. The controllercan indicate the virtual barrier on an occupancy grid stored on a memorystorage element operable with the controller. For example, in someimplementations, the controller can indicate on the occupancy grid thatunknown cells adjacent to traversable cells (e.g., a row or a column oftraversable cells, two or more traversable cells that form a row orcolumn of cells) in the first area are non-traversable (e.g., that thenon-traversable cells define a virtual barrier). As a result, thenon-traversable cells can form a row or column of non-traversable cells.Other methods of defining the boundary that do not rely on the occupancygrid may also be used. In some cases, the controller can indicate thattraversable cells in the first area adjacent to unknown cells are nownon-traversable.

At operation 470, the robot executes a second cleaning operation toclean the second area without traversing the virtual barrier. Forexample, the robot can clean the second area without traversing avirtual barrier marking the perimeter of the first area. The controllercan issue an instruction to the robot to execute the second cleaningoperation. The second cleaning operation can be an execution of acoverage behavior. To prevent itself from entering the first region, thecontroller can prevent the robot from traversing the virtual barrierestablished in operation 468.

In some examples, a user may desire to set a virtual boundary for therobot. For example, the user may want to keep the robot out of aparticular room or area. Allowing the user to establish the location ofa virtual boundary can provide the advantage of giving the useradditional control of where the robot cleans. In some implementations,the controller can receive instructions from a user to confinenavigation of the robot within an area of the environment. The user candeliver the instructions by triggering sensors (e.g., pushing one ormore buttons) on the robot. In some cases, the user can use a mobiledevice, such as a smartphone, tablet, or other computing device, todeliver the instructions to the controller using a wireless connectionto establish the location of the virtual barrier. The user may seek tokeep the robot from exiting a room through a doorway, and thus caninstruct the controller to generate a virtual barrier located at thedoorway that prevents the robot from exiting through the doorway. Insome implementations, the user enters information to restrict robotmovement through the robot's user interface.

In the example illustrated in FIGS. 6A to 6C, a user places a robot(e.g., the robot 200 described with respect to FIGS. 3A to 3B) in anenvironment 502 before the robot 200 executes a cleaning operation toclean the floor surface 10 of the environment 502. A controller (e.g.,the controller 390) of the robot 200 generates an occupancy grid 518corresponding to the environment 502. In this example, the user may wishto sequentially clean a first room 504 during a first cleaning operationand a second room 506 during a second cleaning operation. The user mayseek to have the robot 200, in one cleaning operation, clean the firstroom 504 without cleaning the second room 506 in the environment 502.

Referring to FIG. 6A, the user positions the robot 200 in theenvironment 502 such that the back side 202A of the body 202 of therobot 200 is placed parallel to a wall 512 and a doorway 517 in theenvironment 502, as shown in portion 521. The user then issues aninstruction to the controller 390 to generate a virtual barrier 516 inthe occupancy grid 518, as shown in portion 523. In some examples, thevirtual barrier 516 may manifest in the occupancy grid 518 as a line(e.g., a row or column) of non-traversable cells based on the initialposition and orientation of the robot 200 in the environment 502. Thevirtual barrier 516 can be parallel to the back side 202A of the robot200.

In some cases, the virtual barrier 516 passes through the back side 202Aof the robot 200. In other cases, the virtual barrier 516 intersects therobot body, e.g., the virtual barrier passes through the lights 242 aand 242 b enabling the user to align the lights with the location of thevirtual barrier. The lights 242 a and 242 b therefore may serve asvisual indicators of the location of the virtual barrier 516. Thevirtual barrier 516 can prevent the robot 200 from passing from thefirst room 504 through a doorway 517 into the room 506 of theenvironment 502. In some implementations, the robot can be placed in thedoorway 517 so that the controller generates the virtual barrier 516that prevents the robot 200 from passing through the doorway 517.

After the user has completed its instructions to the controller togenerate the virtual barrier 516, without repositioning the robot, theuser can initiate the cleaning operation in the room 504. When the robot200 starts the cleaning operation, now referring to FIG. 6B, the robot200 can turn 90 degrees such that the forward drive direction F of therobot 200 is parallel to the virtual barrier 516 (e.g., as shown in theportion 523 of FIG. 6B). The 90-degree turn ensures that, in thecoverage behavior, the robot 200 executes the first row of the cornrowpattern adjacent to the virtual barrier 516. In some cases, driftminimally affects the first row of the cornrow pattern, so having therobot 200 execute the first row parallel to the virtual barrier 516 isadvantageous because the robot 200 is not likely to cross the virtualbarrier. In addition, the 90-degree turn prevents the 180-degree turnsin the cornrow pattern from occurring at the virtual barrier 516. Afterthe robot 200 turns, the robot 200 can then proceed to execute acoverage behavior (e.g., performing the cornrow pattern). In some cases,the robot 200 may move in the forward drive direction a short distance(e.g., 2 to 5 cm, 5 to 10 cm, 10 to 15 cm) and then turn 90 degrees toalign a lateral side of the robot 200 to be parallel with the virtualbarrier 516. For example, the robot may move forward by the distancebetween the visual indicators (e.g., the lights 242 a, 242 b) and theback side of the robot 200.

The user can provide the instructions to the robot 200 through a numberof methods and mechanisms. The controller can respond to a trigger thatplaces the robot 200 in a handshake or virtual barrier mode where thecontroller is prepared to populate an occupancy grid with the virtualbarriers. When the robot 200 is in the handshake mode, the controllerplaces the virtual barrier 516. The trigger can be, for example, theuser simultaneously compressing the bumper 210 of the robot 200 andpressing the clean button 240 of the robot 200 while robot is either onor off the ground (e.g., as determined by sensing the ground usingappropriate sensors, as described herein). The user may manipulate therobot 200 in other ways as well to toggle the trigger and initiate thehandshake mode. For instance, the user may trigger the accelerometer orgyroscope of the robot 200 by shaking the robot 200, and upon sensingthe shake, the robot 200 enters the handshake mode to place one or bothof the virtual barriers. In some cases, the user may instruct the robot200 using a mobile device. The user may position the robot 200 in theenvironment and then instruct the robot 200 by, for example, using anapplication loaded on the mobile device. In some implementations, thecontroller, upon placing the robot into the handshake mode, awaitsfurther instructions from the user to generate the virtual barrier. Theuser can issue another instruction—after instructing the robot to enterthe handshake mode—to place the virtual barrier 516 in the occupancygrid.

In some implementations, the controller can generate a second virtualbarrier that may be perpendicular or otherwise angled relative to thefirst virtual barrier 516. The second virtual barrier may restrict therobot from a region that may be a difficult-to-clean area or an areawith fragile furniture or household items. The second virtual barriermay be a virtual barrier of non-traversable cells in the occupancy grid518. The virtual barrier can be generated based on the initial positionand/or orientation of the robot 200. In some examples, the first andsecond virtual barriers can form L-shape of non-traversable cells. Insome cases, the second virtual barrier may coincide with the right side202R or the left side 202L of the robot body 202. In other examples, thecontroller may generate the second virtual barrier such that the secondvirtual barrier passes through the light 242 a or the light 242 b. Thecontroller can generate the second virtual barrier in response to theinstruction to generate the first virtual barrier. In otherimplementations, the controller generates the second virtual barrier inresponse to a second instruction from the user to generate a virtualbarrier. In some cases, the controller places the second virtual barrierwhen the user places the robot into the handshake mode for a first timeor for a second time. In cases where the controller generates twovirtual barriers, the robot 200 may initiate the cleaning operationwithout turning to become parallel with the virtual barrier 516. In somecases, the robot 200 may initiate the cleaning operation by turning suchthat the robot 200 is parallel to the generated virtual barrier.

Referring to FIG. 6C, a flow chart 560 illustrates a method for a robotto generate a virtual barrier based on an instruction from a user. Theflow chart includes user operations 565 corresponding to operationsexecuted by the user and robot operations 570 corresponding tooperations executed by the robot.

At operation 572, the user positions the robot within an environment.The position of the robot will serve as both the starting location ofthe robot and the location of the virtual barrier. As such, the user canposition the robot such that a feature on the robot is aligned with(e.g., parallel to) an edge in the environment that the user does notwant to the robot to cross (e.g., across which a virtual barrier is tobe erected). For example, as described herein, the feature can be lightson the robot or a surface of the robot body. In some cases, the user maywish to create two (e.g., perpendicular) virtual barriers so that therobot does not cross two edges in the environment, and in such cases,the robot may have two features, each indicating a position andorientation of a virtual barrier.

At operation 574, the user instructs the robot to enter a virtualbarrier mode. The user may issue this instruction using any of themethods described herein, or any other appropriate method, that triggerthe robot to enter the handshake mode. At operation 576, a controller ofthe robot receives the instruction and places the robot into the virtualbarrier mode.

At operation 578, the user instructs the robot to generate a virtualbarrier. The instruction to generate the virtual barrier can be theinstruction to place the robot into the virtual barrier mode (e.g., toplace the robot into the handshake mode). In some cases, the user mayissue a subsequent instruction—apart from the instruction to place therobot into the virtual barrier mode—to generate the virtual barrier. Forexample, the user may trigger additional sensors to send theinstructions to create the virtual barrier.

At operation 580, the controller receives the instructions to create thevirtual barrier. The controller may receive the instructions by sensingthat the sensors have been triggered in the manners described herein. Insome cases, the robot may include a wireless transceiver that allows thecontroller to communicate with a mobile device to receive instructionsfrom the user.

At operation 582, the controller generates the virtual barrier. Forexample, the controller may define cells in an occupancy grid as beingpart of the virtual barrier. For example, the virtual barrier cancorrespond to one or more cells that are designated as non-traversable.In some implementations, the virtual barrier may not be defined in termsof cells in the occupancy grid. Instead, the virtual barrier may bedefined based on coordinates on the occupancy grid or some otherfeatures that are within, or outside of, the context of the occupancygrid. For example, the virtual barrier is defined based on the initialorientation and position of the robot. Measurements of these orientationmay be obtained, e.g., based on signals output from the gyroscope housedwithin the body of the robot. The controller may know the initiallocation of the robot, or a part thereof, in the occupancy gridimmediately following the handshake. Using this information, namely theorientation and the initial location, the controller may create thevirtual barrier by defining a boundary (e.g., a straight line) on theoccupancy grid (or elsewhere) that the robot cannot cross. In some casesthe controller may generate more than one virtual barrier as describedherein. In some examples, the user can select the length of the virtualbarrier by providing the controller with appropriate parameters eitherdirectly on the robot or through a remote interface. For example, theuser can select a 3 to 5-foot (0.9 to 1.6 meter) barrier length toprohibit the robot from passing through a door. In some examples, theuser can instruction the robot place a full length barrier of cells in arow/column for sub-dividing an open space. In another case, the user canselect a rectangular region surrounding the robot, forming four virtualbarriers that the robot should not cross.

At operation 584, the controller can provide a visual indication ofgeneration of the virtual barrier. For example, the controller caninstruct lights on the robot to illuminate or can issue an audiblealert.

At operation 586, the user instructs the robot to clean the environment.The user can instruct the robot to clean by pressing the clean button onthe robot or by using the mobile device to remotely control the robot.The virtual barrier can be displayed on a map displayed on a user'smobile device.

At operation 588, the controller receives the instruction to clean theenvironment without traversing the virtual barrier. The robot canexecute the instructions to clean the environment by executing cornrowbehavior or other movement patterns to cover a floor surface of theenvironment. The controller may instruct the robot to turn such that theforward drive direction of the robot is parallel to the virtual barrier.In some implementations, the controller instructs the robot to turnsubstantially 90 degrees to orient the robot parallel to the virtualbarrier.

While the examples illustrated in FIGS. 6A to 6C have been described touse the robot 200 described in FIGS. 3A to 3B, the robot 100 and othermobile robots having other configurations can readily implement themethods described herein. The robot used to implement the methods ofFIGS. 6A to 6C can have other distinctive surfaces or features that theuser can use as a reference for the placement of the virtual barrier.While the robot 200 has been described to be a square robot, in somecases, the robot implementing the methods described herein may be around or a triangular robot. As a result, the virtual barrier generatedmay be tangential to a back surface of the robot. The robot can alsohave additional or alternative sensors that the user can trigger inorder to instruct the controller to generate the virtual barrier.

The methods described herein to generate a virtual barrier can occurbefore the robot initiates a cleaning operation. In someimplementations, the robot begins the cleaning operation and navigatesaround an environment before the robot generates the virtual barrier oradditional virtual barrier(s) may be generated during cleaning. Forexample, the robot can detect features, markers, or other visual indicialocated in the environment and respond to the features by populating theoccupancy grid with a virtual barrier or by otherwise defining one ormore virtual barrier(s) that the robot cannot cross. An example of suchan indicator can be a sticker or tag that is machine identifiable andcan be positioned in the environment.

The robot 100, as described earlier, includes the camera 109 to imagewall surfaces of the environment. Referring to FIG. 7A, in an example,the robot 100 is executing a coverage behavior along the floor surface10 of an environment 602 (e.g., as shown in portion 621) as part of acleaning operation. Executing the cornrow pattern, the robot 100 followsa path 604 and designates cells in an occupancy grid 606 as traversableor non-traversable (e.g., as shown in portion 623). The environment 602includes a first room 607 and a second room 608. The robot 100 isexecuting the cleaning operation to clean the first room 607. Along thepath 604, the robot 100 can sense (e.g., a capture an image of) a wallsurface 609 of the environment 602 using the camera 109.

At a point 604 a along the path 604, the robot 100 detects markers 610a, 610 b located on the wall surface 609. A user may place the markers610 a, 610 b on the wall surface 609 to restrict the robot 100 fromentering a region of the environment. For example, the markers 610 a,610 b may indicate that a traversable area by the robot 100 should bemarked as non-traversable in the occupancy grid 606 of the robot 100.The markers 610 a, 610 b can be fixed to the wall surface 609 through,for example, an adhesive or static backing. The markers 610 a, 610 b mayinclude suction cups that can generate a suction force to fix the cupsto surfaces of the environment 602. In some implementations, the markers610 a, 610 b include infrared dots or ink that may be detectable by aninfrared transceiver of the robot 100 without being human perceptibleunder normal conditions.

In the example shown in FIGS. 7A to 7B, the feature is a doorway 611that connects the first room 607 to the second room 608. The user placesthe markers 610 a, 610 b approximately 1 m to 2 m above the floorsurface on the wall surface 609 so that the robot 100 can detect themarkers 610 a, 610 b using the camera 109, which is angled upward towardthe wall surface 609. In some examples, the markers 610 a, 610 b can beabove the doorway or placed on the inside of the doorway. For example,the user may place the markers 610 a, 610 b along a horizontal surfaceabove the doorway and facing downward toward the floor surface so thatthe upward angled camera 109 can detect the markers 610 a, 610 b. Theplacement of the markers 610 a, 610 b adjacent the doorway 611 canestablish the location of a virtual barrier and make sure that the robot100 only cleans the first room 607 and does not enter the second room608.

Along the path 604 at the point 604 a, now also referring to FIG. 7B,the robot 100 detects the markers 610 a, 610 b on the wall surface 609using the camera 109. The markers 610 a, 610 b include distinctivefeatures or machine-readable information that can be sensed by thecamera 109. Thus, some markers 610 a, 610 b can indicate the location ofa virtual barrier while other markers can be used to relay other typesof information to the robot 100. The machine-readable information orfeature can represent a name of a location corresponding to thestructure or obstacle in the environment. In some cases, themachine-readable information can represent a name of a locationcorresponding to the structure or obstacle in the environment. Thefeature or machine-readable information may be a color, image, or othercharacteristic that can be detected by the camera 109. And, in someimplementations, the camera 109 may be responsive to radiation outsideof the visible light range and therefore may also be able to detect, forexample, infrared characteristics of the markers 610 a, 610 b. While thecamera 109 has been described as the sensor to detect the markers 610 a,610 b, in some implementations, the robot 100 may use other sensors todetect the markers 610 a, 610 b, such as ultrasonic, infrared, and otherdirectional beam sensors.

The distinctive features may indicate attributes of the environment 602and/or the wall surface 609. These features may be used foridentification purposes in addition or as an alternative to establishinga virtual barrier. The memory storage element 395 can include a libraryof reference features to which the controller 390 can compare the imagedmarkers 610 a, 610 b. The controller 390 can then determine whether themarkers 610 a, 610 b include features within the library of referencefeatures.

In some examples, the features of the markers 610 a, 610 b may indicatethat the environment 602 through which the robot 100 is navigating is aparticular room, such as a kitchen, a bathroom, a bedroom, a livingroom, etc. For example, the markers 610 a, 610 b may include arefrigerator icon that indicates that the first room 607 is a kitchen,and a television icon that indicates that the second room is a livingroom. In some cases, the markers 610 a, 610 b may indicate a type ofstructure exists between the markers 610 a, 610 b. For example, in somecases, the markers 610 a, 610 b may indicate that the doorway 611 liesin between the markers 610 a, 610 b. In other cases, the markers 610 a,610 b may be placed in the environment 602 such that the robot does notenter a difficult-to-clean area or an area with fragile furniture orhousehold items. The markers 610 a, 610 b may be placed on lamps,furniture, or other household objects that can be imaged by the camera109. For example, one type of marker could establish a keep-out zone ofa predefined distance from the marker (e.g., 0.25 m to 0.5 m, 0.5 m to 1m, 1 m to 1.5 m). The markers 610 a, 610 b can have a particular colorfor specific attributes, or a specific image for particular rooms. Insome implementations, the markers 610 a, 610 b may include distinctiveimages to serve as the distinctive features of the markers 610 a, 610 b.

The distinctive features may also be names of the room that the markers610 a, 610 b mark, names of the obstacles that the markers 610 a, 610 bmark, or names of the locations that the markers 610 a, 610 b mark. Forexample, in implementations where the robot 100 has maps generated fromprevious cleaning operations, the markers 610 a, 610 b may indicate thatthe robot 100 is in the kitchen, and the robot 100 may then use a mapcorresponding to the kitchen that was previously generated. In somecases, the robot 100 may not begin a cleaning operation until it detectsthe markers 610 a, 610 b. When the robot 100 detects the markers 610 a,610 b, the robot 100 can begin a cleaning operation based on theinformation from the markers 610 a, 610 b. The information provided bythe distinctive features may be transmitted to a mobile device so that auser can see the information and select operations of the robot 100based on the information.

The controller can post-process the images generated of the markers 610a, 610 b before identifying the markers 610 a, 610 b. For example, thecontroller may rectify the images using an affine transformation or someother computer vision process for image rectification. Aftertransforming the images of the markers 610 a, 610 b, the controller cancompare the images to stored reference images in, for example, thelibrary of reference features on the memory storage element 395 of therobot 100 in order to confirm that the robot 100 has detected themarkers 610 a, 610 b. The comparison can also allow the controller 390to determine the type of information provided by the markers 610 a, 610b (e.g., attributes of the environment 602 and the wall surface 609). Insome implementations, the markers 610 a, 610 b each can have multipleportions conveying different types of information. One portion of eachof the markers 610 a, 610 b can indicate the type of the first room 607that the robot 100 is currently in, and another portion of each of themarkers 610 a, 610 b can indicate the type of the second room 608connected to the doorway 611.

In examples where the markers 610 a, 610 b are used to establish virtualbarriers, upon detecting the markers 610 a, 610 b and confirming thatthe robot has detected the markers 610 a, 610 b, the robot 100 candesignate a virtual barrier 612 (e.g., a set of non-traversable cells)in the occupancy grid 606 based on the positions of the markers 610 a,610 b. For example, the controller can compute a line 614 that passesthrough both the marker 610 a and the marker 610 b. The line 614 isparallel to the virtual barrier 612 that the controller designates inthe occupancy grid 606. While the virtual barrier 612 in the occupancygrid 606 is shown to be in between the markers 610 a, 610 b, in someimplementations, the virtual barrier 612 generated from sensing themarkers 610 a, 610 b may span a greater length than the line 614 thatconnects the markers 610 a, 610 b.

The markers 610 a, 610 b can indicate to the robot 100 that the doorway611 exists in between the markers 610 a, 610 b. In such cases, uponfinishing the cleaning operation of the first room 607, the robot 100can, in a subsequent cleaning operation, move to the virtual barrier 612and begin a subsequent cleaning operation to clean the second room 608.The virtual barrier 612 may persist, but, instead of cleaning the firstroom 607 on the right side of the virtual barrier 612, the robot 100cleans the second room 608.

The robot 100 can continue to clean the first room 607 within the boundsof the virtual barrier 612 and the physical wall surface 609 until oneor more conditions are met. The one or more conditions can include, forexample, covering a percentage of the defined area and/or otherconditions described herein.

In some implementations, the robot 100 may remember the virtual barrier612 in a subsequent cleaning operation (e.g., in a persistent occupancygrid). The user may remove the markers 610 a, 610 b after the firstcleaning operation when the robot 100 detects the markers 610 a, 610 b,and the virtual barrier 612 as part of the first cleaning operationpersists. The robot 100, for example, stores the virtual barrier 612 anduses it for the subsequent cleaning operation. Upon starting thesubsequent cleaning operation in the first room 607, the robot 100remains in the first room 607 and does not proceed through the doorway611 to the second room 608.

Referring to FIG. 7C, a flow chart 660 illustrates a method of usingmarkers in an environment to instruct a robot to generate a virtualbarrier in an occupancy grid stored on the robot. The flow chart 660includes user operations 665 corresponding to operations executed by theuser and robot operations 670 corresponding to operations executed bythe robot.

At operation 672, the user places the markers in the environment. Theuser can place the markers such that they flank a specific feature inthe environment the user does not want the user to traverse, such as adoorway, threshold, or other opening. The markers may be placed on asurface in the environment to identify a room item. The surface may bethe surface of a wall, obstacle, or other object in the environment.

At operation 674, the user instructs the robot to begin a first cleaningoperation. The user may use a mobile device or may depress a button onthe robot to instruct the robot to begin the first cleaning operation.

At operation 676, a controller of the robot receives the instruction tobegin the first cleaning operation. At operation 678, the robot executesthe first cleaning operation. In some cases, the controller begins thefirst cleaning operation, by, for example, instructing the robot tobegin the cleaning operation. During the cleaning operation, the robotmay execute the cornrow pattern, as described herein, or some othermovement pattern to cover a floor surface of the environment.

At operation 680, the robot detects the markers in the environment. Thecontroller can use a camera, ultrasonic sensor, or some other sensor onthe robot to detect the markers. In some cases, as described herein, thecamera may detect a color, image, or other distinctive feature of themarkers. The controller can receive image data from the cameracorresponding to the detection of the markers.

At operation 682, the controller determines whether the detected markersare virtual barrier markers. The controller may also post-process theimage data of the detected markers and make a determination of whetherthe image data correspond to reference images that the controller mayexpect from detecting the markers. The controller may compare the imagedata to reference images in a library stored on a memory storage elementoperable with the controller. The controller can determine whether thedetected markers indicate a virtual barrier, a location, or otherinformation about the environment.

If the controller determines that the detected markers are virtualbarrier markers, at operation 684, the controller generates a virtualbarrier in an occupancy grid that, for example, corresponds to thelocation of the detected markers. The virtual barrier, as describedherein, can correspond to a set of non-traversable cells to be marked onthe occupancy grid. In some cases, the length or width of thenon-traversable barrier may depend on distinctive features detected onthe markers. If the controller determines that the detected marker isnot a virtual barrier marker, at operation 686, the controller storesdata related to the detected marker in the occupancy grid. The data maybe, for example, a name of the room, a name of the location of thedetected markers. In some implementations, the controller may determinethat the controller has misidentified the detected markers and that thedetected markers do not indicate information about the environment. Insome examples, the controller may determine that the detected markersindicate both a virtual barrier and data related to the name of the roomor the location of the detected markers.

At operation 688, the controller determines whether the first cleaningoperation is complete. The controller can evaluate whether the robot hasmet one or more conditions as described herein. If the controllerdetermines that the first cleaning operation is complete, at operation690, the robot completes the first cleaning operation. If the controllerdetermines that the first cleaning operation is not complete, atoperation 692, the robot continues the first cleaning operation. Thecontroller can instruct the robot to continue the first cleaningoperation. The robot can then continue to detect markers in theenvironment, or in some cases, the robot continues the first cleaningoperation and then completes the first cleaning operation withoutdetecting additional markers and proceeds to operation 690.

In some implementations, the controller may store the virtual barrier tobe used in a subsequent cleaning operation. As a result, at operation694, the user may remove the markers from the environment. In someimplementations, the user may keep the markers in the environment, andsubsequent detections of the markers by the camera of the robot canincrease the confidence that the camera has detected the markers.

Then, at operation 696, the user can instruct the robot to begin asecond cleaning operation. In some cases, the user instructs the robotto begin the second cleaning operation in the environment that the robotcleaned during the first cleaning operation. In other cases, the userinstructs the robot to begin the cleaning operation in anotherenvironment. At operation 698, the controller receives the instructionto begin the second cleaning operation using the occupancy gridgenerated during the first cleaning operation. The controller theninstructs the robot to begin the second cleaning operation. If the robotbegins the second cleaning operation in the environment cleaned duringoperations 678 and 692, the robot cleans the same areas and does notcross the virtual barrier. If the robot begins the second cleaningoperation in another environment, the robot can clean an area differentthan the area cleaned during the first cleaning operation, and thevirtual barrier effectively prevents the robot from returning the areacleaned during operation 678 and 692.

While the examples illustrated in FIGS. 7A to 7C have been describedwith respect to robot 100 described in FIGS. 2A to 2B, other mobilerobots having other appropriate configurations can implement the methodsdescribed herein. For example, the robot 200 can include a camera thatcan execute the functions described herein. In some implementations, thecamera 109 can capture images that the controller can use to identifygeometric features characteristic of doorways (e.g., a rectangularopening that extends from the floor through a portion of the wall). Thecontroller can then place a virtual barrier corresponding to thelocation of the doorway geometry detected by the camera 109.

The robot 100, as described herein, includes the infrared transceiver118 to detect infrared radiation emitted into the environment. Referringto FIG. 8A, a gateway beacon 701 is located on the floor surface 10 ofan environment 702 including a first room 704 and a second room 706(e.g., as shown in portion 721 of FIG. 8A). A doorway 707 separates thefirst room 704 from the second room 706. The gateway beacon 701 emits aninfrared gateway beam 708 detectable by the infrared transceiver 118. Auser can place the gateway beacon 701 in the environment 702 and canorient the gateway beacon 701 such that the gateway beam 708 points in aspecific direction. For example, the gateway beam 708 can be directedacross the length of the doorway 707.

While cleaning the first room 704, the robot 100 may execute a cornrowpattern in the form of a path 709. As the robot 100 navigates about thefirst room 704 along the path 709, the robot 100 may detect the gatewaybeam 708 as the robot 100 passes by the gateway beam 708 using, forexample, the infrared transceiver 118. The robot 100 can detect thegateway beam 708 and interpret the locations where the robot 100 detectsthe gateway beam 708 as a virtual barrier 710 (e.g., a set ofnon-traversable cells) in an occupancy grid 712 of the robot 100 (e.g.,as shown in portion 723 of FIG. 8A). Although FIG. 8A shows that thepath 709 passes near the gateway beam 708, in other implementations, thepath 709 may pass through the gateway beam 708. The gateway beacon 701and its gateway beam 708 thus prevents the robot 100 from passingthrough the doorway 707.

Referring to FIG. 8B, the robot 100, in a subsequent cleaning operation,the robot 100 can store the location of the virtual barrier 710 in, forexample, memory or on a remote computing device as part of a persistentmap (e.g., as shown in the portion 723 of FIG. 8B). As a result, whenthe gateway beacon 701 placed in the environment 702 in FIG. 8A isremoved from the environment for subsequent cleaning operations, therobot 100 can still prevent itself from crossing the virtual barrier710. In some cases, the robot 100 can be placed in the first room 704and re-clean the first room 704 without crossing the virtual barrier 710into the second room 706. In other cases, the robot 100 can be placed inthe second room 706 and can clean the second room 706 without cleaningthe first room 704 again.

Referring to FIG. 8C, a flow chart 760 illustrates a method of using agateway beacon in an environment to instruct a robot to generate avirtual barrier in an occupancy grid stored on the robot. The flow chart760 includes user operations 765 corresponding to operations executed bythe user and robot operations 770 corresponding to operations executedby the robot.

At operation 772, the user places the gateway beacon in the environment.The user can place the gateway beacon on the floor surface of theenvironment such that the gateway beam marks a specific feature orlocation in the environment that the user does not want the robot totraverse, such as a doorway, threshold, or other opening.

At operation 774, the user instructs the robot to begin a first cleaningoperation. The user may use a mobile device or depress a button on therobot to instruct the robot to begin the first cleaning operation.

At operation 776, the controller of the robot receives the instructionto begin the first cleaning operation. At operation 778, the controllerbegins the first cleaning operation.

At operation 780, a transceiver of the robot detects the gateway beam inthe environment. The transceiver can be an infrared transceiver.

At operation 782, the controller generates a virtual barrier in anoccupancy grid or other persistent map. The virtual barrier, asdescribed herein, can correspond to a line of non-traversable cells tobe marked on the occupancy grid. In some implementations, the virtualbarrier can be a set of coordinates that define a line or curve in anoccupancy grid. In some cases, the length or width of thenon-traversable barrier may depend on the strength of the signal thatthe robot senses as it detects the gateway beam in operation 780.

At operation 784, the controller completes the first cleaning operation.The controller can complete the first cleaning operation by, forexample, determining that the robot has met one or more conditions suchas, for example, covering a percentage of the defined area and/orfulfilling other conditions described herein.

In some implementations, the robot may store the virtual barrier in apersistent map to be used in a subsequent cleaning operation. As aresult, at operation 786, the user may remove the gateway beacon fromthe environment. Then, at operation 788, the user can instruct the robotto begin a second cleaning operation. In some cases, the user instructsthe robot to begin the second cleaning operation in the environment thatthe robot cleaned during the first cleaning operation. In other cases,the user instructs the robot to begin the cleaning operation in anotherenvironment. At operation 790, the robot begins the second cleaningoperation using the occupancy grid generated during the first cleaningoperation. If the robot begins the second cleaning operation in theenvironment cleaned during operation 778, the robot generally cleans thesame areas and does not cross the virtual barrier. If the robot beginsthe second cleaning operation in another environment, the robot canclean an area different than the area cleaned during the first cleaningoperation, and the virtual barrier effectively prevents the robot fromreturning to the area cleaned during operation 778.

While the examples illustrated in FIGS. 8A to 8C have been described touse the robot 100 described in FIGS. 2A to 2B, other mobile robotshaving other appropriate configurations can implement the methodsdescribed herein. For example, the robot 200 can include an infraredtransceiver that can execute the functions described herein.

While the virtual barriers generated herein have been described to bestraight walls, in some implementations, the virtual barriers can becircular. For example, placing the robot into the handshake modedescribed with respect to FIGS. 6A to 6C can cause the controller togenerate a substantially circular virtual barrier that can, for example,restrict a robot to a circular area rug. In some cases, the user caninstruct the controller to generate a circular virtual barrier using amobile computing device that can communicate with the communicationssystem of the robot. In some cases, the robot may continue the cleaningoperation in the circular area until the controller has determined thatthe robot has fulfilled one or more conditions, such as, for example,covering a percentage of the defined area and/or fulfilling otherconditions described herein. In other examples, the virtual barrier canestablish a circular keep out zone.

The controller may use the virtual barriers to divide an environmentinto two or more regions to be covered separately. For example, thevirtual barrier may divide the environment into two regions, where oneregion corresponds to for example, a kitchen, bathroom, a carpet, etc.,and a second region corresponds to a bedroom, a living room, hardwoodfloor, etc. The controller can instruct the robot to clean the firstregion in one cleaning operation and then clean the second region in asubsequent cleaning operation. In some cases, the controller caninstruct the robot to clean one region in a deeper cleaning mode wherethe robot will repeat a cleaning operation multiple times in the region.In some implementations, the user can label the individual regions ofthe environment as particular rooms in a house, such as a kitchen,bedroom, or bathroom. As described herein, the controller can alsodetect features in the markers 610 a, 610 b that can allow thecontroller to associate labels with regions of the environment. The usercan then use the mobile computing device to instruct the robot to cleana labeled region. The user can also instruct the robot to keep out of alabeled region while the robot cleans another labeled region.

While in at least some of the examples described herein, the virtualbarriers were stored in an occupancy grid used by the robot forlocalization, the virtual barriers could be stored in other types ofmaps used by the robot for localization and navigation.

The system can be controlled or implemented, at least in part, using oneor more computer program products, e.g., one or more computer programstangibly embodied in one or more information carriers, such as one ormore non-transitory machine-readable media, for execution by, or tocontrol the operation of, one or more data processing apparatus, e.g., aprogrammable processor, a computer, multiple computers, and/orprogrammable logic components.

A computer program can be written in any form of programming language,including compiled or interpreted languages, and it can be deployed inany form, including as a stand-alone program or as a module, component,subroutine, or other unit suitable for use in a computing environment.

Actions associated with implementing all or part of the controlmechanism described herein can be performed by one or more programmableprocessors executing one or more computer programs to perform thefunctions described herein. All or part of the control mechanismdescribed herein can be implemented using special purpose logiccircuitry, e.g., an FPGA (field programmable gate array) and/or an ASIC(application-specific integrated circuit).

Processors suitable for the execution of a computer program include, byway of example, both general and special purpose microprocessors, andany one or more processors of any kind of digital computer. Generally, aprocessor will receive instructions and data from a read-only storagearea or a random access storage area or both. Elements of a computerinclude one or more processors for executing instructions and one ormore storage area devices for storing instructions and data. Generally,a computer will also include, or be operatively coupled to receive datafrom, or transfer data to, or both, one or more machine-readable storagemedia, such as mass PCBs for storing data, e.g., magnetic,magneto-optical disks, or optical disks. Machine-readable storage mediasuitable for embodying computer program instructions and data includeall forms of non-volatile storage area, including by way of example,semiconductor storage area devices, e.g., EPROM, EEPROM, and flashstorage area devices; magnetic disks, e.g., internal hard disks orremovable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks.

Elements of different implementations described herein may be combinedto form other embodiments not specifically set forth above. Elements maybe left out of the structures described herein without adverselyaffecting their operation. Furthermore, various separate elements may becombined into one or more individual elements to perform the functionsdescribed herein.

1. (canceled)
 2. An autonomous cleaning robot comprising: a drive systemconfigured to move the autonomous cleaning robot across a floor surfaceof an environment; a cleaning system positioned on a bottom portion ofthe autonomous cleaning robot, the cleaning system configured to cleanthe floor surface; and a controller configured to execute instructionsto perform operations comprising: creating one or more virtual barriersthat divide the floor surface into a plurality of regions; operating theautonomous cleaning robot in accordance with a first behavioral modewith respect to a first region of the plurality of regions; andoperating the autonomous cleaning robot in accordance with a secondbehavioral mode with respect to a second region of the plurality ofregions, wherein the second behavioral mode differs from the firstbehavioral mode.
 3. The autonomous cleaning robot of claim 2, whereinthe autonomous cleaning robot comprises a wireless transceiverconfigured to communicate with a mobile computing device; and whereinthe operations comprise receiving, at the wireless transceiver and fromthe mobile computing device, data indicative of one or more userinstructions.
 4. The autonomous cleaning robot of claim 3, whereincreating the one or more virtual barriers comprises creating the one ormore virtual barriers based on the data indicative of the one or moreuser instructions.
 5. The autonomous cleaning robot of claim 3, whereinthe operations comprise associating the first behavioral mode with thefirst region and/or associating the second behavioral mode with thesecond region based on the data indicative of the one or more userinstructions.
 6. The autonomous cleaning robot of claim 2, wherein theoperations comprise associating a first label with the first region or afirst structure within the first region and/or associating a secondlabel with the second region or a second structure within the secondregion; and associating the first behavioral mode with the first regionbased on the first label and/or associating the second behavioral modewith the second region based on the second label.
 7. The autonomouscleaning robot of claim 6, wherein the first label and/or the secondlabel is indicative of a type of room.
 8. The autonomous cleaning robotof claim 6, wherein the first label and/or the second label isindicative of a fragility of the first structure and/or the secondstructure.
 9. The autonomous cleaning robot of claim 6, wherein thefirst label and/or the second label is indicative of a difficulty toclean the first region and/or the second region.
 10. The autonomouscleaning robot of claim 6, wherein the autonomous cleaning robotcomprises a wireless transceiver configured to communicate with a mobilecomputing device; wherein the operations comprise receiving, at thewireless transceiver and from the mobile computing device, dataindicative of one or more user instructions; and wherein associating thefirst label with the first region or the first structure within thefirst region is based on the data indicative of the one or more userinstructions and/or associating the second label with the second regionor the second structure within the second region is based on the dataindicative of the one or more user instructions.
 11. The autonomouscleaning robot of claim 6, wherein the operations comprise detecting oneor more features within the environment; and wherein associating thefirst label with the first region or the first structure within thefirst region is based on the one or more detected features and/orassociating the second label with the second region or the secondstructure within the second region is based on the one or more detectedfeatures.
 12. The autonomous cleaning robot of claim 2, wherein thefirst behavioral mode or the second behavioral mode comprises a deepcleaning mode.
 13. The autonomous cleaning robot of claim 2, wherein thefirst behavioral mode comprises keeping the autonomous cleaning robotout of the first region or wherein the second behavioral mode compriseskeeping the autonomous cleaning robot out of the second region.
 14. Theautonomous cleaning robot of claim 2, wherein at least one of the one ormore virtual barriers is non-linear.
 15. The autonomous cleaning robotof claim 2, wherein the one or more virtual barriers define an enclosedshape.
 16. A mobile computing device comprising: a wireless transceiverconfigured to communicate with an autonomous cleaning robot movableabout a floor surface of an environment; a user input device operable bya user to provide one or more user instructions, the one or more userinstructions comprising instructions to initiate creating one or morevirtual barriers that divide the floor surface into a plurality ofregions; and one or more processors configured to execute operationscomprising: receiving the one or more user instructions; andtransmitting, from the wireless transceiver of the mobile computingdevice to the autonomous cleaning robot, data indicative of the one ormore user instructions such that, during an autonomous operation of theautonomous cleaning robot, the autonomous cleaning robot is operated inaccordance with a first behavioral mode with respect to a first regionof the plurality of regions; and is operated in accordance with a secondbehavioral mode with respect to a second region of the plurality ofregions, wherein the second behavioral mode differs from the firstbehavioral mode.
 17. The mobile computing device of claim 16, whereinthe one or more user instructions comprise instructions to associate thefirst behavioral mode with the first region and/or associate the secondbehavioral mode with the second region.
 18. The mobile computing deviceof claim 16, wherein the first region or a first structure within thefirst region is associated with a first label, and/or wherein the secondregion or a second structure within the second region is associated witha second label.
 19. The mobile computing device of claim 18, wherein thefirst label and/or the second label is indicative of at least one of (i)a type of a room, (ii) a fragility of the first structure and/or thesecond structure, or (iii) a difficulty to clean the first region and/orthe second region.
 20. The mobile computing device of claim 18, whereinthe one or more user instructions comprise instructions to associate thefirst label with the first region or the first structure within thefirst region and/or wherein the one or more user instructions compriseinstructions to associate the second label with the second region or thesecond structure within the second region.
 21. The mobile computingdevice of claim 16, wherein the first behavioral mode or the secondbehavioral mode comprises a deep cleaning mode.
 22. The mobile computingdevice of claim 16, wherein the first behavioral mode comprises keepingthe autonomous cleaning robot out of the first region or wherein thesecond behavioral mode comprises keeping the autonomous cleaning robotout of the second region.